VXLAN Anycast Gateway
Published: 2023-01-27
One of the strengths of VXLAN is that you can distribute the default gateway IP-address across multiple nodes for active-active forwarding. This is a massive leap forward when compared to active-passive HSRP and VRRP-based solutions from previous generations. We will configure a VXLAN with VRRP lab and then a VXLAN with Anycast Gatway lab to compare the two.
Later in the article we will look at the two modes of performing VXLAN-based inter-vlan routing, namely Asymmetric IRB and Symmetric IRB. We will configure each and compare their benefits and drawbacks. I'm really happy with the way this article turned out, I hope you are too once you've finished reading it.
This article builds on concepts that I already explored in VXLAN L2VPN and VXLAN L3VPN, so you may benefit the most by reading those first.
VXLAN with VRRP
Let's start with a VXLAN topology that utilizes VRRP. The diagram below describes the lab topology we will be using in this article. We have two VLANs and subnets, BLUE and RED. We have three VXLAN-switches, SW2-SW4. R1 is a "core" router without any VXLAN capabilities, it is only forwarding traffic between the SW2, SW3 and SW4 loopbacks.
-
SW2 is the VRRP master for vlan BLUE thanks to its 250 priority being the highest. SW3 (priority 200) and SW4 (priority 150) are in backup state, ready to take over if SW2 dies. Part of the master role of SW2 is responding to ARP requests for the Virtual IP-address 10.0.10.1. The reply is sent using the VRRP virtual MAC-address 0000.5e00.1111.
SW2 also sends out Gratuitous ARPs every 30 seconds, announcing the VIP and vMAC. This is flooded out on all local switchports aswell as all VTEPs in the VXLAN flood list. -
For the RED vlan, SW3 is the VRRP master thanks to its 200 priority. SW2 only has a priority of 50 here so it's relegated to backup duty for this virtual IP. It is therefore the job of SW3 to respond to ARP requests for 10.0.20.1, using the VRRP virtual MAC-address 0000.5e00.2222.
I could have kept things simple by having SW2 as the VRRP master for both subnets but I believe this setup has more instructive value.
VXLAN with VRRP Configuration
If you want to see how this was implemented, you can review the configuration below.
service routing protocols model multi-agent
!
vlan 10
name BLUE
!
vlan 20
name RED
!
interface Ethernet1
no switchport
ip address 10.0.12.2/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet21
switchport access vlan 10
!
interface Ethernet22
switchport access vlan 20
!
interface Loopback0
ip address 10.0.0.2/32
!
interface Vlan10
ip address 10.0.10.2/24
vrrp 1 priority-level 250
vrrp 1 ipv4 10.0.10.1
!
interface Vlan20
ip address 10.0.20.2/24
vrrp 2 priority-level 50
vrrp 2 ipv4 10.0.20.1
!
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 10 vni 10
vxlan vlan 20 vni 20
vxlan flood vtep 10.0.0.3 10.0.0.4
!
ip routing
!
router ospf 1
redistribute connected
service routing protocols model multi-agent
!
vlan 10
name BLUE
!
vlan 20
name RED
!
interface Ethernet1
no switchport
ip address 10.0.13.3/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet31
switchport access vlan 10
!
interface Ethernet32
switchport access vlan 20
!
interface Loopback0
ip address 10.0.0.3/32
!
interface Vlan10
ip address 10.0.10.3/24
vrrp 1 priority-level 200
vrrp 1 ipv4 10.0.10.1
!
interface Vlan20
ip address 10.0.20.3/24
vrrp 2 priority-level 200
vrrp 2 ipv4 10.0.20.1
!
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 10 vni 10
vxlan vlan 20 vni 20
vxlan flood vtep 10.0.0.2 10.0.0.4
!
ip routing
!
router ospf 1
redistribute connected
service routing protocols model multi-agent
!
vlan 10
name BLUE
!
vlan 20
name RED
!
interface Ethernet1
no switchport
ip address 10.0.14.4/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet41
switchport access vlan 10
!
interface Ethernet42
switchport access vlan 20
!
interface Loopback0
ip address 10.0.0.4/32
!
interface Vlan10
ip address 10.0.10.4/24
vrrp 1 priority-level 150
vrrp 1 ipv4 10.0.10.1
!
interface Vlan20
ip address 10.0.20.4/24
vrrp 2 priority-level 150
vrrp 2 ipv4 10.0.20.1
!
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 10 vni 10
vxlan vlan 20 vni 20
vxlan flood vtep 10.0.0.2 10.0.0.3
!
ip routing
!
router ospf 1
redistribute connected
interface Ethernet2
no switchport
ip address 10.0.12.1/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet3
no switchport
ip address 10.0.13.1/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet4
no switchport
ip address 10.0.14.1/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
ip routing
!
router ospf 1
router-id 10.0.0.1
redistribute connected
The configuration consists of a rather basic VXLAN L2VPN with statically configured flood lists for BUM-traffic. On top of that a VLAN-interface has been configured for each BLUE and RED vlan. The PCs in the topology have IP-address 10.0.XX.1 set as their default gateway, with XX being the vlan ID of BLUE or RED.
If we follow an ICMP packet from PC41 to PC42 and back, the full path is as shown below:
Starting with PC41, the ICMP Request is sent to its default gateway. SW4 has learned the 0000.5e00.1111 MAC-address via VTEP 10.0.0.2 VNI 10, so it performs VXLAN encapsulation and forwards the packet to SW2 via R1.
SW2 perform inter-vlan routing and send the packet back to SW4 as it has learned the PC42 MAC-address via VTEP 10.0.0.4 VNI 20. This VXLAN encapsulation is a must as R1 have no idea about the BLUE or RED vlans.
The reply from PC42 to PC41 is routed by SW3 as it is the active default gateway for the RED vlan.
VXLAN with VRRP output
Let us examine the output of our various SW2-SW4 tables to see what they
SW2#show vrrp
Vlan10 - Group 1
State is Master
Virtual IPv4 address is 10.0.10.1
Virtual MAC address is 1111
Vlan20 - Group 2
State is Backup
Virtual IPv4 address is 10.0.20.1
Virtual MAC address is 2222
SW2#show mac add
Vlan Mac Address Ports
---- ----------- ----
10 0003 Vx1 # SW3
10 0041 Vx1 # PC41
20 2222 Vx1 # VRRP VIP
20 0042 Vx1 # PC42
SW2#show vxlan add
VLAN Mac Address Prt VTEP
---- ----------- --- ----
10 0003 Vx1 10.0.0.3
10 0041 Vx1 10.0.0.4
20 2222 Vx1 10.0.0.3
20 0042 Vx1 10.0.0.4
SW2#show arp
Address Age (sec) Addr Interface
10.0.10.41 0:00:23 0041 Vlan10, Vxlan1
10.0.20.1 0:00:07 2222 Vlan20, Vxlan1
10.0.20.42 0:00:20 0042 Vlan20, Vxlan1
SW3#show vrrp
Vlan10 - Group 1
State is Backup
Virtual IPv4 address is 10.0.10.1
Virtual MAC address is 1111
Vlan20 - Group 2
State is Master
Virtual IPv4 address is 10.0.20.1
Virtual MAC address is 2222
SW3#show mac add
Vlan Mac Address Ports
---- ----------- ----
10 1111 Vx1
10 0041 Vx1
20 0002 Vx1
20 0042 Vx1
SW3#show vxlan add
VLAN Mac Address Prt VTEP
---- ----------- ---- ----
10 1111 Vx1 10.0.0.2
10 0041 Vx1 10.0.0.4
20 0002 Vx1 10.0.0.2
20 0042 Vx1 10.0.0.4
SW3#show arp
Address Age (sec) Addr Interface
10.0.10.1 0:00:05 1111 Vlan10, Vxlan1
10.0.10.41 0:00:19 0041 Vlan10, Vxlan1
10.0.20.42 0:00:20 0042 Vlan20, Vxlan1
SW4#show vrrp
Vlan10 - Group 1
State is Backup
Virtual IPv4 address is 10.0.10.1
Virtual MAC address is 1111
Vlan20 - Group 2
State is Backup
Virtual IPv4 address is 10.0.20.1
Virtual MAC address is 2222
SW4#show mac add
Vlan Mac Address Ports
---- ----------- ----
10 1111 Vx1 # VRRP VIP
20 2222 Vx1 # VRRP VIP
10 0003 Vx1 # SW3
10 0041 Et2 # PC41
20 0002 Vx1 # SW2
20 0042 Et3 # PC42
SW4#show vxlan add
VLAN Mac Address Prt VTEP
---- ----------- --- ----
10 1111 Vx1 10.0.0.2
10 0003 Vx1 10.0.0.3
20 2222 Vx1 10.0.0.3
20 0002 Vx1 10.0.0.2
SW4#show arp
Address Age (sec) Addr Interface
10.0.10.1 0:00:05 1111 Vlan10, Vxlan1
10.0.20.1 0:00:07 2222 Vlan20, Vxlan1
I have shortened the MAC-addresses to make this output easier to read.
Focusing on the SW4 output, it has learned the MAC-address of the VRRP VIPs, PC41 and PC42 but also the MAC-addresses of SW2 and SW3. This is because SW2 and SW3 source traffic from their local Vlan10/Vlan20 MAC-address when performing inter-vlan routing.
VRRP (and HSRP) Conclusion
While VRRP is a useful protocol for provoding First-hop redundancy, it is always implemented in a active-passive fashion. The reason for this is that a switched network do not like when multiple devices try to use the same MAC-address. As the topology above grows, more and more traffic will be sent to SW2 and SW3 for inter-vlan routing. Their routing capacity becomes a bottleneck for the potential growth of the network.
You could spread the VRRP Masters out across multiple nodes, but this is only a bandaid as traffic exiting that vlan relies on the throughput of a single switch. Additionaly, you still have the issue of east-west inter-vlan traffic tromboning across the network even though the endpoints may be connected to the same switch.
With this information in mind, let's examine the VXLAN Anycast Gateway solution.
VXLAN with Anycast Gateway (AGW)
Instead of using VRRP, which forces all inter-vlan traffic through the VRRP Master node, we can use VXLAN AGW to enable active-active inter-vlan routing across SW2, SW3 and SW4.
Before we continue, let's answer why this technology is only possible in VXLAN-enabled networks. The answer is that the nature of VXLAN separates the VXLAN-switches into their own flooding domains. By only flooding packets out on local switchports and not to any remote VTEPs, the VXLAN-switch guarantees that the virtual MAC-address is isolated on the local VXLAN island.
This is why downstream switches should not be connected to multiple VXLAN-switches, as that would join two VXLAN flooding domains. An exception here is MLAG-enabled VXLAN-switches, but this is out of scope for this article. Another exception is STP which would likely block one of the uplinks, but again, out of scope. My point is that each VXLAN island should stay isolated.
Let's review the necessary configuration for Anycast Gateway:
ip virtual-router mac-addres 5001.0000.0234
!
interface Vlan10
ip address virtual 10.0.10.1/24
!
interface Vlan20
ip address virtual 10.0.20.1/24
First we assign a common virtual MAC-address to be used on all VXLAN-switches, in this case 5001.0000.0234. Secondly we use the ip address virtual x.x.x.x/yy command to configure the Virtual IP-address on each VLAN-interface. The virtual keyword tells the switch to use the virtual MAC-address we configured instead of the MAC-address phsyically assigned to the VLAN-interface.
When configured, each VXLAN AGW will send out a Gratuitous ARP every 30 seconds on all local switchports, announcing the IP-address and MAC-address combination (10.0.10.1, 5001.0000.0234). When a device send traffic to its default gateway, the destination MAC-address is 5001.0000.0234, allowing whichever VXLAN-switch that receives it to process it and perform routing. When PC41 and PC42 now wants to communicate, SW4 can route it directly without having to forward to SW2 or SW3!
The model we just used is called Asymmetric Integrated Routing & Bridging (IRB). This is the simplest model. Another model Symmetric IRB also exist. We will compare these two models next.
Asymmetric vs Symmetric IRB
While the Asymmetric IRB model is simple in its execution it has one drawback; the VLAN configuration on all VXLAN-switches must be the same. If you add a VLAN to the network, it must be configured on all VXLAN-switches. If you add a VLAN-interface to that VLAN you just added, it must be configured on all VXLAN-switches.
The reason for this is that for inter-vlan routing to work correctly, each switch must have an active entry for the destination host in its ARP table, MAC-address (and VXLAN-address table if the destination is not locally connected). Without this information the packet may have to be flooded. Even worse, if the outbound VLAN-interface is not configured on the switch the destination cannot be resolved and the packet must be dropped or matched against some other route, blackholing the packet.
While this drawback may not seem like such a big deal, this puts a limit on your naximum network size as MAC-addresses must be learned on switches that may not have any locally connected devices on that VLAN. The size of your network is now limited to the maximum MAC-address table size of your smallest switch.
The benefit of this model is that the configuration is simple and looks the same on all switches, which makes automation easier. If you expect around 10k end-devices in your network and your smallest switch can fit 32k entries then you're likely fine.
Oh, I almost forgot. The model is called asymmetric because the jobs performed by the ingress VTEP and the egress VTEP are different when forwarding. Let's say PC21 on SW2 on vlan BLUE wants to communicate with PC42 on SW4 in vlan RED. In this instance, SW2 is the ingress VTEP so it performs inter-vlan routing, VXLAN-encapsulates the frame and sends it to SW4. SW4, the egress VTEP, only decapsulates the VXLAN packet and then forwards the frame to PC42, no routing performed.
When PC42 sends traffic to PC21, SW4 is now the ingress VTEP and must perform inter-vlan routing while SW2 only decapsulates and forwards the frame to PC21. The forwarding process is asymmetrical, hence the name.
Symmetric IRB Theory
The way the symmetric model works: when the ingress VTEP VXLAN-encapsulates a frame, the VNI value is mapped to a VRF instead of a VLAN. The egress VTEP will then perform a routing lookup inside that VRF before forwarding the packet. Because both the ingress and egress VTEPs perform routing lookups, the model is said to be symmetrical.
This model allows the network to grow larger as we no longer need to keep all MAC-addresses on all VXLAN-switches. Each VLAN is configured only on switches that have locally connected devices.
To demonstrate this we will need to alter the topology, as shown below. I also choose to enable EVPN with this example as you are likely to want to run EVPN together with Symmetric IRB. You may implement Symmetric IRB without EVPN, but it requires static routes that use VTEP nexthops, which I covered in my VXLAN L3VPN article.
Symmetric IRB Topology
An example of Symmetric IRB forwarding in our topology above would be traffic between PC21 and PC42. Because SW2 only has vlan BLUE configured and SW4 only has vlan RED configured, none of them can perform asymmetric inter-vlan routing. So when SW2 forwards the frame to SW4, VNI value 1 is set in the VXLAN header. This tells SW4 that the payload packet belongs to vrf GREEN and that it must perform a routing lookup to find the outbound interface to PC42.
If we instead look at packets sent from PC31 to PC42, SW3 can perform asymmetric inter-vlan routing as it has a local VLAN-interface in both VLANs. The return path, however, require SW4 to use symmetric inter-vlan routing as it does not have the VLAN-interface BLUE configured.
For SW2 to know that it should add VNI 1 to the VXLAN header, it must learn the 10.0.20/24 route from SW4. In this topology we use EVPN Type-5 routes to advertise these routes. I will this cover this below.
Symmetric IRB Configuration
With the model and topology explained, let's review the configuration needed for this to work:
service routing protocols model multi-agent
!
vlan 10
name BLUE
!
vrf instance GREEN
!
interface Ethernet1
no switchport
ip address 10.0.12.2/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet21
switchport access vlan 10
!
interface Loopback0
ip address 10.0.0.2/32
!
interface Vlan10
vrf GREEN
ip address virtual 10.0.10.1/24
!
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 10 vni 10
vxlan vrf GREEN vni 1
!
ip virtual-router mac-address 50:01:00:00:02:34
!
ip routing
ip routing vrf GREEN
!
arp aging timeout default 290
!
router bgp 65000
maximum-paths 2
neighbor EVPN peer group
neighbor EVPN remote-as 65000
neighbor EVPN update-source Loopback0
neighbor EVPN send-community
neighbor 10.0.0.3 peer group EVPN
neighbor 10.0.0.4 peer group EVPN
!
vlan 10
rd 65000:10
route-target both 65000:10
redistribute learned
!
address-family evpn
neighbor EVPN activate
!
address-family ipv4
no neighbor EVPN activate
!
vrf GREEN
rd 65000:1
route-target import evpn 65000:1
route-target export evpn 65000:1
redistribute connected
!
router ospf 1
redistribute connected
service routing protocols model multi-agent
!
vlan 10
name BLUE
!
vlan 20
name RED
!
vrf instance GREEN
!
interface Ethernet1
no switchport
ip address 10.0.13.3/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet31
switchport access vlan 10
!
interface Ethernet32
switchport access vlan 20
!
interface Loopback0
ip address 10.0.0.3/32
!
interface Vlan10
vrf GREEN
ip address virtual 10.0.10.1/24
!
interface Vlan20
vrf GREEN
ip address virtual 10.0.20.1/24
!
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 10 vni 10
vxlan vlan 20 vni 20
vxlan vrf GREEN vni 1
!
ip virtual-router mac-address 50:01:00:00:02:34
!
ip routing
ip routing vrf GREEN
!
arp aging timeout default 290
!
router bgp 65000
maximum-paths 2
neighbor EVPN peer group
neighbor EVPN remote-as 65000
neighbor EVPN update-source Loopback0
neighbor EVPN send-community
neighbor 10.0.0.2 peer group EVPN
neighbor 10.0.0.4 peer group EVPN
!
vlan 10
rd 65000:10
route-target both 65000:10
redistribute learned
!
vlan 20
rd 65000:20
route-target both 65000:20
redistribute learned
!
address-family evpn
neighbor EVPN activate
!
address-family ipv4
no neighbor EVPN activate
!
vrf GREEN
rd 65000:1
route-target import evpn 65000:1
route-target export evpn 65000:1
redistribute connected
!
router ospf 1
redistribute connected
service routing protocols model multi-agent
!
vlan 20
name RED
!
vrf instance GREEN
!
interface Ethernet1
no switchport
ip address 10.0.14.4/29
ip ospf network point-to-point
ip ospf area 0.0.0.0
!
interface Ethernet42
switchport access vlan 20
!
interface Loopback0
ip address 10.0.0.4/32
!
interface Vlan20
vrf GREEN
ip address virtual 10.0.20.1/24
!
interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 20 vni 20
vxlan vrf GREEN vni 1
!
ip virtual-router mac-address 50:01:00:00:02:34
!
ip routing
ip routing vrf GREEN
!
arp aging timeout default 290
!
router bgp 65000
maximum-paths 2
neighbor EVPN peer group
neighbor EVPN remote-as 65000
neighbor EVPN update-source Loopback0
neighbor EVPN send-community
neighbor 10.0.0.2 peer group EVPN
neighbor 10.0.0.3 peer group EVPN
!
vlan 20
rd 65000:20
route-target both 65000:20
redistribute learned
!
address-family evpn
neighbor EVPN activate
!
address-family ipv4
no neighbor EVPN activate
!
vrf GREEN
rd 65000:1
route-target import evpn 65000:1
route-target export evpn 65000:1
redistribute connected
!
router ospf 1
redistribute connected
This is the full configuration for SW2-SW4 in this lab topology. The major addition is vrf GREEN which is now configured on all three VXLAN-switches. We also added EVPN configuration to help advertise routes between the SW2-SW4 nodes.
The BGP adjacencies are formed between loopbacks, a common practice with iBGP. OSPF is in charge of keeping a loop-free topology that allows for inter-loopback communication while BGP is in charge of exchanging "customer" routes. This is a design that is used in most Service-Provider networks. This topology is known as a BGP-free core network design as R1 does not participate in BGP. but let's not go into that now.
The redistribute learned BGP command under vlan 10 allow switches to advertise MAC-addresses they have learned on their switchports.
The redistribute connected BGP command under vrf GREEN allow switches to advertise the BLUE and/or RED interface prefixes. This is what enables SW4 to know where to send traffic for the 10.0.10/24 subnet, for example.
The maximum-paths 2 BGP command enable the receiver to install multiple paths for a prefix and perform ECMP loadbalancing.
The arp aging timeout default 290 global command is covered in my VXLAN L2VPN article. A very important command when EVPN is used.
Symmetric IRB Output
With the most important bits covered, let's go through the forwarding table output. The full output is huge, so I have decided to keep the boxes closed by default, and instead focus on parts of the output as we go along. If you want to browse through the full output now or after you've finished the article, feel free to do so.
SW2#show mac add
Vlan Mac Address Type Ports Moves Last Move
---- ----------- ---- ----- ----- ---------
10 aabb.cc00.0021 DYNAMIC Et2 1 3:21:06 ago
10 aabb.cc00.0031 DYNAMIC Vx1 1 0:31:13 ago
4094 5001.00ca.0004 DYNAMIC Vx1 1 1:00:59 ago
4094 5001.00ca.0003 DYNAMIC Vx1 1 1:00:59 ago
SW2#show vxlan addr
VLAN Mac Address Type Prt VTEP Moves Last Move
---- ----------- ---- --- ---- ----- ---------
10 aabb.cc00.0031 EVPN Vx1 10.0.0.3 1 0:31:19 ago
4094 5001.00ca.0004 EVPN Vx1 10.0.0.4 1 1:01:05 ago
4094 5001.00ca.0003 EVPN Vx1 10.0.0.3 1 1:01:05 ago
SW2#show arp vrf GREEN
Address Age (sec) Hardware Addr Interface
10.0.10.21 0:01:17 aabb.cc00.0021 Vlan10, Ethernet2
10.0.10.31 - aabb.cc00.0031 Vlan10, Vxlan1
SW2#show ip route vrf GREEN
B I 10.0.10.31/32 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
C 10.0.10.0/24 is directly connected, Vlan10
B I 10.0.20.32/32 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
B I 10.0.20.42/32 [200/0] via VTEP 10.0.0.4 VNI 1 router-mac 5001.00ca.0004
B I 10.0.20.0/24 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
SW2#show bgp evpn
Network Next Hop Metric LocPref Weight Path
* > RD: 65000:10 mac-ip aabb.cc00.0021
- - - 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0021 10.0.10.21
- - - 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0031
10.0.0.3 - 100 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0031 10.0.10.31
10.0.0.3 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0032
10.0.0.3 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0032 10.0.20.32
10.0.0.3 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0042
10.0.0.4 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0042 10.0.20.42
10.0.0.4 - 100 0 i
* > RD: 65000:10 imet 10.0.0.2
- - - 0 i
* > RD: 65000:10 imet 10.0.0.3
10.0.0.3 - 100 0 i
* > RD: 65000:20 imet 10.0.0.3
10.0.0.3 - 100 0 i
* > RD: 65000:20 imet 10.0.0.4
10.0.0.4 - 100 0 i
* > RD: 65000:1 ip-prefix 10.0.10.0/24
- - - 0 i
* RD: 65000:1 ip-prefix 10.0.10.0/24
10.0.0.3 - 100 0 i
* > RD: 65000:1 ip-prefix 10.0.20.0/24
10.0.0.3 - 100 0 i
* RD: 65000:1 ip-prefix 10.0.20.0/24
10.0.0.4 - 100 0 i
SW2#show bgp evpn detail
BGP routing table entry for mac-ip aabb.cc00.0021
Route Distinguisher: 65000:10
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0021 10.0.10.21
Route Distinguisher: 65000:10
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0031
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0031 10.0.10.31
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 10
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0032
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0032 10.0.20.32
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 20
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0042
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0042 10.0.20.42
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0004
VNI: 20
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for imet 10.0.0.2
Route Distinguisher: 65000:10
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.2
BGP routing table entry for imet 10.0.0.3
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.3
BGP routing table entry for imet 10.0.0.3
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.3
BGP routing table entry for imet 10.0.0.4
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.4
BGP routing table entry for ip-prefix 10.0.10.0/24
Route Distinguisher: 65000:1
Paths: 2 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best, redistributed (Connected)
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0002
VNI: 1
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 1
BGP routing table entry for ip-prefix 10.0.20.0/24
Route Distinguisher: 65000:1
Paths: 2 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 1
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0004
VNI: 1
SW3#show mac add
Vlan Mac Address Type Ports Moves Last Move
---- ----------- ---- ----- ----- ---------
10 aabb.cc00.0021 DYNAMIC Vx1 1 0:31:13 ago
10 aabb.cc00.0031 DYNAMIC Et2 1 3:21:07 ago
20 aabb.cc00.0032 DYNAMIC Et3 1 3:21:07 ago
20 aabb.cc00.0042 DYNAMIC Vx1 1 1:01:24 ago
4094 5001.00ca.0002 DYNAMIC Vx1 1 1:00:59 ago
4094 5001.00ca.0004 DYNAMIC Vx1 1 1:00:59 ago
SW3#show vxlan addr
VLAN Mac Address Type Prt VTEP Moves Last Move
---- ----------- ---- --- ---- ----- ---------
10 aabb.cc00.0021 EVPN Vx1 10.0.0.2 1 0:31:18 ago
20 aabb.cc00.0042 EVPN Vx1 10.0.0.4 1 1:01:30 ago
4094 5001.00ca.0002 EVPN Vx1 10.0.0.2 1 1:01:05 ago
4094 5001.00ca.0004 EVPN Vx1 10.0.0.4 1 1:01:05 ago
SW3#show arp vrf GREEN
Address Age (sec) Hardware Addr Interface
10.0.10.21 - aabb.cc00.0021 Vlan10, Vxlan1
10.0.10.31 0:00:51 aabb.cc00.0031 Vlan10, Ethernet2
10.0.20.32 0:01:11 aabb.cc00.0032 Vlan20, Ethernet3
10.0.20.42 - aabb.cc00.0042 Vlan20, Vxlan1
SW3#show ip route vrf GREEN
B I 10.0.10.21/32 [200/0] via VTEP 10.0.0.2 VNI 1 router-mac 5001.00ca.0002
C 10.0.10.0/24 is directly connected, Vlan10
B I 10.0.20.42/32 [200/0] via VTEP 10.0.0.4 VNI 1 router-mac 5001.00ca.0004
C 10.0.20.0/24 is directly connected, Vlan20
SW3#show bgp evpn
Network Next Hop Metric LocPref Weight Path
* > RD: 65000:10 mac-ip aabb.cc00.0021
10.0.0.2 - 100 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0021 10.0.10.21
10.0.0.2 - 100 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0031
- - - 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0031 10.0.10.31
- - - 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0032
- - - 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0032 10.0.20.32
- - - 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0042
10.0.0.4 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0042 10.0.20.42
10.0.0.4 - 100 0 i
* > RD: 65000:10 imet 10.0.0.2
10.0.0.2 - 100 0 i
* > RD: 65000:10 imet 10.0.0.3
- - - 0 i
* > RD: 65000:20 imet 10.0.0.3
- - - 0 i
* > RD: 65000:20 imet 10.0.0.4
10.0.0.4 - 100 0 i
* > RD: 65000:1 ip-prefix 10.0.10.0/24
- - - 0 i
* RD: 65000:1 ip-prefix 10.0.10.0/24
10.0.0.2 - 100 0 i
* > RD: 65000:1 ip-prefix 10.0.20.0/24
- - - 0 i
* RD: 65000:1 ip-prefix 10.0.20.0/24
10.0.0.4 - 100 0 i
SW3#show bgp evpn det
BGP routing table information for VRF default
Router identifier 10.0.0.3, local AS number 65000
BGP routing table entry for mac-ip aabb.cc00.0021
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0021 10.0.10.21
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0002
VNI: 10
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0031
Route Distinguisher: 65000:10
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0031 10.0.10.31
Route Distinguisher: 65000:10
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0032
Route Distinguisher: 65000:20
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0032 10.0.20.32
Route Distinguisher: 65000:20
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0042
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0042 10.0.20.42
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0004
VNI: 20
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for imet 10.0.0.2
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.2
BGP routing table entry for imet 10.0.0.3
Route Distinguisher: 65000:10
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.3
BGP routing table entry for imet 10.0.0.3
Route Distinguisher: 65000:20
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.3
BGP routing table entry for imet 10.0.0.4
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.4
BGP routing table entry for ip-prefix 10.0.10.0/24
Route Distinguisher: 65000:1
Paths: 2 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best, redistributed (Connected)
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 1
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0002
VNI: 1
BGP routing table entry for ip-prefix 10.0.20.0/24
Route Distinguisher: 65000:1
Paths: 2 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best, redistributed (Connected)
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 1
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0004
VNI: 1
SW4#show mac add
Vlan Mac Address Type Ports Moves Last Move
---- ----------- ---- ----- ----- ---------
20 aabb.cc00.0032 DYNAMIC Vx1 1 1:01:24 ago
20 aabb.cc00.0042 DYNAMIC Et3 1 4:07:37 ago
4094 5001.00ca.0002 DYNAMIC Vx1 1 1:00:59 ago
4094 5001.00ca.0003 DYNAMIC Vx1 1 1:00:59 ago
SW4#show vxlan addr
VLAN Mac Address Type Prt VTEP Moves Last Move
---- ----------- ---- --- ---- ----- ---------
20 aabb.cc00.0032 EVPN Vx1 10.0.0.3 1 1:01:30 ago
4094 5001.00ca.0002 EVPN Vx1 10.0.0.2 1 1:01:05 ago
4094 5001.00ca.0003 EVPN Vx1 10.0.0.3 1 1:01:05 ago
SW4#show arp vrf GREEN
Address Age (sec) Hardware Addr Interface
10.0.20.32 - aabb.cc00.0032 Vlan20, Vxlan1
10.0.20.42 0:01:32 aabb.cc00.0042 Vlan20, Ethernet3
SW4#show ip route vrf GREEN
B I 10.0.10.21/32 [200/0] via VTEP 10.0.0.2 VNI 1 router-mac 5001.00ca.0002
B I 10.0.10.31/32 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
B I 10.0.10.0/24 [200/0] via VTEP 10.0.0.2 VNI 1 router-mac 5001.00ca.0002
B I 10.0.20.32/32 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
C 10.0.20.0/24 is directly connected, Vlan20
SW4#show bgp evpn
Network Next Hop Metric LocPref Weight Path
* > RD: 65000:10 mac-ip aabb.cc00.0021
10.0.0.2 - 100 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0021 10.0.10.21
10.0.0.2 - 100 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0031
10.0.0.3 - 100 0 i
* > RD: 65000:10 mac-ip aabb.cc00.0031 10.0.10.31
10.0.0.3 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0032
10.0.0.3 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0032 10.0.20.32
10.0.0.3 - 100 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0042
- - - 0 i
* > RD: 65000:20 mac-ip aabb.cc00.0042 10.0.20.42
- - - 0 i
* > RD: 65000:10 imet 10.0.0.2
10.0.0.2 - 100 0 i
* > RD: 65000:10 imet 10.0.0.3
10.0.0.3 - 100 0 i
* > RD: 65000:20 imet 10.0.0.3
10.0.0.3 - 100 0 i
* > RD: 65000:20 imet 10.0.0.4
- - - 0 i
* > RD: 65000:1 ip-prefix 10.0.10.0/24
10.0.0.2 - 100 0 i
* RD: 65000:1 ip-prefix 10.0.10.0/24
10.0.0.3 - 100 0 i
* > RD: 65000:1 ip-prefix 10.0.20.0/24
- - - 0 i
* RD: 65000:1 ip-prefix 10.0.20.0/24
10.0.0.3 - 100 0 i
SW4#show bgp evpn detail
BGP routing table information for VRF default
Router identifier 10.0.0.4, local AS number 65000
BGP routing table entry for mac-ip aabb.cc00.0021
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0021 10.0.10.21
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0002
VNI: 10
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0031
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0031 10.0.10.31
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 10
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0032
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0032 10.0.20.32
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 20
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0042
Route Distinguisher: 65000:20
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for mac-ip aabb.cc00.0042 10.0.20.42
Route Distinguisher: 65000:20
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
BGP routing table entry for imet 10.0.0.2
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.2
BGP routing table entry for imet 10.0.0.3
Route Distinguisher: 65000:10
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:10
TunnelEncap:tunnelTypeVxlan
VNI: 10
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.3
BGP routing table entry for imet 10.0.0.3
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.3
BGP routing table entry for imet 10.0.0.4
Route Distinguisher: 65000:20
Paths: 1 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best
Extended Community:
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
VNI: 20
PMSI Tunnel: Ingress Replication, Tunnel ID: 10.0.0.4
BGP routing table entry for ip-prefix 10.0.10.0/24
Route Distinguisher: 65000:1
Paths: 2 available
Local
10.0.0.2 from 10.0.0.2 (10.0.0.2)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0002
VNI: 1
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 1
BGP routing table entry for ip-prefix 10.0.20.0/24
Route Distinguisher: 65000:1
Paths: 2 available
Local
- from - (0.0.0.0)
Origin IGP, metric -, localpref -, weight 0, valid, local, best, redistributed (Connected)
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0004
VNI: 1
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal
Extended Community:
Route-Target-AS:65000:1
TunnelEncap:tunnelTypeVxlan EvpnRouterMac:5001.00ca.0003
VNI: 1
Most of the output in the boxes above is from the show bgp evpn detail command. We will go through the output in steps below. Let's start by examining how routes are received and installed in our forwarding tables. Let's start by looking at the 10.0.20.0/24 route that was advertised by SW3 and SW4 as a Type-5 EVPN route:
SW2#show bgp evpn rd 65000:1 detail
BGP routing table entry for ip-prefix 10.0.20.0/24
Route Distinguisher: 65000:1
Paths: 2 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, ECMP
Extended Community:
Route-Target-AS: 65000:1
TunnelEncap: tunnelTypeVxlan
EvpnRouterMac: 5001.00ca.0003
VNI: 1
Local
10.0.0.4 from 10.0.0.4 (10.0.0.4)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, ECMP
Extended Community:
Route-Target-AS: 65000:1
TunnelEncap: tunnelTypeVxlan
EvpnRouterMac: 5001.00ca.0004
VNI: 1
SW2#show ip route vrf GREEN
C 10.0.10.0/24 is directly connected, Vlan10
B I 10.0.20.0/24 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
via VTEP 10.0.0.4 VNI 1 router-mac 5001.00ca.0004
The show bgp evpn command in the output show the routes in the BGP RIB, displaying parameters that were sent with the prefix. One of the route parameters is EvpnRouterMac which requires a long explanation, not covered here.
The short explanation is that 5001.00ca.0003 is a physical MAC-address used by SW3. So when SW2 or SW4 send VXLAN-encapsulated traffic to SW3 for VNI 1, they must set the destination MAC-address of the original frame to 5001.00ca.0003. When SW3 then does the VXLAN decapsulation it will recognize the MAC-address as its own and understand that the packet should be routed. I explain this better and in more detail in my VXLAN L3VPN article (PC21-PC31 Packet Walk).
The show ip route command show the output after the 10.0.20.0/24 route was installed into the vrf GREEN routing table. Since BGP is configured with maximum-paths 2, the routes from SW3 and SW4 are both installed.
Symmetric IRB and host-routes
The output above leads us to a problem. PC32 (10.0.20.32) lives behind SW3 while PC42 (10.0.20.42) lives behind SW4. When SW2 is to send traffic from PC21 (10.0.10.21) to PC32, how does it know to send it to SW3?
The answer is that SW2 doesn't. In this case SW2 will use ECMP to "randomly" send some flows to SW3 and some flows to SW4. If SW2 forwards a packet destined for PC32 to SW4, SW4 will forward the packet to SW3 with VNI value 20. This means that SW4 has to perform VXLAN decapsulation only to have to VXLAN encapsulate the packet again before sending to SW3. While this is wasteful use of network resources, the switch ASICs generally have no problem doing this.
But we don't want SW2 to guess where traffic should go. To fix this, SW3 and SW4 will apply this configuration:
router bgp 65000
vlan 20
rd 65000:20
route-target both 65000:20
redistribute learned
redistribute host-route
The effect of the redistribute host-route command is that the Type-2 (MAC-IP) route for PC32 advertised by SW3 now includes a L3 VNI, advertising which VRF PC32 belongs to. This command is enabled by default in Arista switches and is therefore hidden, so you can normally only see it when you negate the command with a no redistribute host-route. I chose to explicitly show it in the output above for clarity.
Note: You can also show hidden commands by issuing the show active all command. Be sure to enter vlan 10 under the BGP process before you do so unless you want to watch your terminal scroll for a while.
If we examine the output below we can see a few lines highlighted:
SW2#show bgp evpn route-type mac-ip 10.0.20.32 detail
BGP routing table entry for mac-ip aabb.cc00.0032 10.0.20.32
Route Distinguisher: 65000:20
Paths: 1 available
Local
10.0.0.3 from 10.0.0.3 (10.0.0.3)
Origin IGP, metric -, localpref 100, weight 0, valid, internal, best
Extended Community:
Route-Target-AS:65000:1
Route-Target-AS:65000:20
TunnelEncap:tunnelTypeVxlan
EvpnRouterMac:5001.00ca.0003
VNI: 20
L3 VNI: 1
ESI: 0000:0000:0000:0000:0000
SW2#show ip route vrf GREEN
C 10.0.10.0/24 is directly connected, Vlan10
B I 10.0.20.32/32 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
B I 10.0.20.0/24 [200/0] via VTEP 10.0.0.3 VNI 1 router-mac 5001.00ca.0003
via VTEP 10.0.0.4 VNI 1 router-mac 5001.00ca.0004
The route-target 65000:1 line was added by the redistribute host-route command. Route-targets are typically used to allow the receiving router to discard routes that are not relevant. If SW2 has no VPN instance that wants to import route-target:65000:1 routes then it can safely discard that route to reduce memory usage.
The L3 VNI: 1 line was also added by the redistribute host-route command. It tells SW2 to associate the MAC-IP address combo (aabb.cc00.0032, 10.0.20.32) to VRF GREEN. Thanks to this information, SW2 can install that entry as a host-route in its forwarding table. We can see this route in the VRF GREEN routing table, highlighted last in the output.
So with this /32 host-route in place, SW2 can confidently forward traffic destined for 10.0.20.32 to SW3. If you scroll through the full output above, you can see that 10.0.20.42/32 is also part of the output, allowing SW2 to correctly forward packets with that destination to SW4.
Depending on your needs, you can choose to not redistribute the host-routes. This makes sense if you have a subnet that only exist behind a single VXLAN-switch. Since that switch is the only one advertising the subnet route, there is no need for the other VXLAN-switches to install any host routes. In this scenario you can safely enter no redistribute host-route under that VLAN BGP configuration, saving TCAM space.
Conclusion
This concludes my VXLAN Anycast Gateway article. We have discussed how Anycast active-active forwarding provide higher throughput than the active-passive VRRP/HSRP protocols. We have also compared the Asymmetric IRB and Symmetric IRB models, showing the difference in behavior and configuration.
If you want more to read, please consider other posts in my VXLAN series:
Have a great rest of your day!